format: a GIF moves, and WEBP is the twenty second format - #22
Merged
Conversation
BREAKING: generated bytes change for gif Two formats, one reason. A GIF that cannot animate and a WebP that does not exist both leave a tester unable to answer a question about their own site. ## The GIF moves Reported from use rather than found here: a generated GIF went into an upload form, came back a still picture, and nothing about that said whether the site keeps animations, flattens them to the first frame, or re-encodes them. The plan had declared "animated" and "frame_count" since the format was written, so the manifest could describe an animation and nothing could produce one. Every GIF now carries a marker travelling across the picture in three frames. frames is a setting from 1 to 60. Measured before choosing, at 640x480 and three frames: * a square patch disposed "restore to previous" +329 B * a full width band redrawn each frame +18626 B * the same band, flattened in the base picture +1302 B The first, and the disposal rule it leans on was measured rather than feared: Pillow and ffmpeg compose the frames and find one marker in each with no trail, the Windows Imaging Component and Chromium both count three frames. A separate measurement fixed something that would have cost every file: with no GIF.Config the encoder writes no global colour table and gives every frame its own, so a 12 px square cost 233 B instead of 41 B - 192 B of it a second copy of the palette. The minimum rises from 41 B to 114 B, because the number a format announces has to be one a plain run accepts, and a plain run animates. **--set frames=1 is the way back**, byte for byte. The proof is not a sentence: the golden case gif_64kib_one_frame carries 659e4880..., which is the hash that was pinned under gif_64kib until today. ## WEBP, written rather than borrowed x/image/webp at v0.43.0 holds decode.go and doc.go, so there was no encoder to take. Pure Go encoders exist outside it and were rejected for one reason: their next release would move the hashes in other people's CI, and D11 is ours to keep. The bitstream is 203 lines and compresses nothing on purpose. Three channels declare all 256 literals at eight bits, so the canonical code for a symbol is that symbol, no Huffman tree is built, and a pixel costs exactly 24 bits. The size is then arithmetic that inverts, so the picture GROWS to fill the request instead of being a thumbnail followed by filler - the shape BMP and TIFF have. The padding channel was unverified since the project began, and the note describing it named the wrong problem. Decoder tolerance was never the issue: an unknown chunk at the end passes all six readers. The issue is that every RIFF chunk block costs an EVEN number of bytes, so a file built only of chunks can only have an even length. Hence two stages - a private chunk for the bulk, one to seven bytes after the payload for the rest - and no dead zone at all, which neither PNG nor GIF manage. A negative control is what makes the reader list mean anything: a truncated WebP is accepted by ffprobe and by the Windows Imaging Component, so the evidence rests on the four readers that can say no. Pixels are compared, not merely decoded, in Pillow and in Chromium. ## Two defects found by guards rather than by reading The first encoder built the whole file in memory and carried a comment explaining why it had to - "a Huffman stream is not a sequence of whole bytes". True, and not a reason: the partial byte is one byte of state. The guard measured 67598496 B of allocation for a 67108864 B file and refused. Writing through to the output moved no byte of any file, which the pinned hashes prove. Both new packages then crossed the size and branching ceilings, and are split by job rather than counted upwards - gif and webp each read as container, pixels and bitstream now. Ten mutations, all caught. An eleventh was written and thrown away before it reached the list: mutating the oracle to decode only the first frame breaks nothing, because a valid file decodes fine from frame one.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Two formats, one reason. A GIF that cannot animate and a WebP that does not
exist both leave a tester unable to answer a question about their own site.
This is a breaking change and the version is deliberately untouched. The
### Breakingentry sits under[Unreleased]in the changelog. Raising theversion is the owner's step.
The GIF moves
Reported from use rather than found here: a generated GIF went into an upload
form, came back a still picture, and nothing about that said whether the site
keeps animations, flattens them to the first frame, or re-encodes them. The plan
had declared
animatedandframe_countsince the format was written, so themanifest could describe an animation and nothing could produce one.
Every GIF now carries a marker travelling across the picture in three frames.
framesis a setting from 1 to 60.Measured before choosing, at 640x480 and three frames:
The first. The disposal rule it leans on was measured rather than feared: Pillow
and ffmpeg compose the frames and find one marker in each with no trail, the
Windows Imaging Component and Chromium both count three frames.
A separate measurement fixed something that would have cost every file: with no
GIF.Configthe encoder writes no global colour table and gives every frame itsown, so a 12 px square cost 233 B instead of 41 B - 192 B of it a second copy of
the palette.
The minimum rises from 41 B to 114 B, because the number a format announces
has to be one a plain run accepts, and a plain run animates.
--set frames=1is the way back, byte for byte. The proof is not asentence: the golden case
gif_64kib_one_framecarries659e4880..., which isthe hash that was pinned under
gif_64kibuntil today.WEBP, written rather than borrowed
x/image/webpat v0.43.0 holdsdecode.goanddoc.go, so there was noencoder to take. Pure Go encoders exist outside it and were rejected for one
reason: their next release would move the hashes in other people's CI, and D11
is ours to keep.
The bitstream is 203 lines and compresses nothing on purpose. Three channels
declare all 256 literals at eight bits, so the canonical code for a symbol is
that symbol, no Huffman tree is built, and a pixel costs exactly 24 bits. The
size is then arithmetic that inverts, so the picture GROWS to fill the request
instead of being a thumbnail followed by filler - the shape BMP and TIFF have.
The padding channel was unverified since the project began, and the note
describing it named the wrong problem. Decoder tolerance was never the issue:
an unknown chunk at the end passes all six readers. The issue is that every RIFF
chunk block costs an EVEN number of bytes, so a file built only of chunks can
only have an even length. Hence two stages - a private chunk for the bulk, one
to seven bytes after the payload for the rest - and no dead zone at all,
which neither PNG nor GIF manage.
A negative control is what makes the reader list mean anything: a truncated
WebP is accepted by ffprobe and by the Windows Imaging Component, so the
evidence rests on the four readers that can say no. Pixels are compared, not
merely decoded, in Pillow and in Chromium.
Two defects found by guards rather than by reading
The first encoder built the whole file in memory and carried a comment
explaining why it had to - "a Huffman stream is not a sequence of whole bytes".
True, and not a reason: the partial byte is one byte of state. The guard
measured 67598496 B of allocation for a 67108864 B file and refused. Writing
through to the output moved no byte of any file, which the pinned hashes
prove.
Both new packages then crossed the size and branching ceilings, and are split by
job rather than counted upwards -
gifandwebpeach read as container,pixels and bitstream now.
Checks
Ten mutations, all caught. An eleventh was written and thrown away before it
reached the list: mutating the oracle to decode only the first frame breaks
nothing, because a valid file decodes fine from frame one.
go test ./...green.tools/preflight.py --quickgreen, 11 of 11 - it caughtone thing on the first run, a probe added to
tools/probeswithout a line inthe index.
Both formats opened in a native viewer. Screen references and the website
regenerated, and the diffs read: the format menu grew by one row, the site went
from 21 formats to 22.
Not in this pull request
AVIF. The owner asked for a probe before deciding, the probe is done, and the
verdict with its numbers is written down for the session that will build it.
🤖 Generated with Claude Code